Virtual counseling system and counseling method using the same

ABSTRACT

The present disclosure relates to a virtual counseling system in which a user can virtually receive counseling by inputting query information into a system. A virtual counseling system according to an embodiment of the present disclosure may include an input unit obtaining audio information from a user and generating audio data; a determination unit receiving the audio data through the input unit, determining a type of the audio data, and generating type information on the audio data; and a text data generation unit generating object data by receiving the type information from the determination unit, converting content of the audio data into first text data, and combining the object data and the first text data to generate second text data.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of International PatentApplication No. PCT/KR2019/011796, filed on Sep. 11, 2019, which isbased upon and claims the benefit of priority to Korean PatentApplication No. 10-2018-0115490 filed on Sep. 28, 2018. The disclosuresof the above-listed applications are hereby incorporated by referenceherein in their entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a virtual counseling system in which auser can virtually receive counseling by inputting queries to a system,and a counseling method using the same.

2. Description of Related Art

A call center originally meant a center where a company simply receivedinquiry calls from customers. However, in recent years, call centershave come to perform a function of creating newly added value whileresponding to various requests from customers for information provisionrelated to products and delivery, corporate image enhancement,marketing, and customer service.

The service of the above-described call center is generally providedthrough a telephone. The method of providing service using such atelephone is to receive inquiries and problems of the company, products,delivery, customer service, and other information through a counselorwhen a customer makes a phone call, and then respond with correspondinginformation or measures.

The method of providing a call center service using such a telephone hasa drawback in that a counselor has to always maintain a presence.

Accordingly, there may be a drawback in that the call center service maynot be available after the counselor leaves the office. In addition,when the counselor works overtime, there may be a drawback in that costssuch as labor costs may increase.

Accordingly, in recent years, an Automatic Response System (ARS) hasbeen introduced, wherein data, stored as text, is converted into audioand provided, or various pieces of information are stored as audio, andwhen a customer accesses the system using a telephone, the systeminforms the customer of a method of use so that necessary informationcan be retrieved by audio, and when the necessary information is found,the information is heard as audio.

However, the above-described ARS informs a customer of an informationretrieval method, and then outputs the necessary information upon theinput, which may require a significant amount of time for the customerto be familiar with the above-described information retrieval method. Inaddition, because the ARS is not a conversation-based processing system,the information retrieval method may be very inconvenient to use.

Accordingly, even with the conventional ARS described above, there is adrawback in that it is difficult to accurately and quickly obtain theinformation desired by the customer.

SUMMARY

The present disclosure is directed to addressing an issue associatedwith the related art, and to providing accurate reply data to a questioninput by a user and analyzing a user's emotions so that more detailedcounseling can be made.

In this regard, an aspect of the present disclosure is to provide avirtual counseling system and a counseling method using the same, whichcan accurately grasp a user's intention in asking questions andemotions, and smoothly proceed with counseling based thereon.

Other aspects and advantages of the present disclosure will become moreapparent by reference to the following detailed description of theinvention, claims, and drawings.

A virtual counseling system according to an embodiment of the presentdisclosure may include an input unit obtaining audio information from auser and generating audio data; a determination unit receiving the audiodata through the input unit, determining a type of the audio data, andgenerating type information on the audio data; and a text datageneration unit generating object data by receiving the type informationfrom the determination unit, converting content of the audio data intofirst text data, and combining the object data and the first text datato generate second text data.

A virtual counseling method according to another embodiment of thepresent disclosure may include obtaining audio data from a user;receiving the audio data, determining a type of the audio data, andgenerating type information on the audio data; generating object data byreceiving the type information, converting content of the audio datainto first text data, and combining the object data and the first textdata to generate second text data; and generating feedback data based onthe second text data and providing the feedback data to the user.

According to an aspect of the present disclosure, there is provided avirtual counseling system and a counseling method using the same, whichcan accurately grasp the user's emotional state and intention in askingquestions and, and proceed with smooth counseling based thereon.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the overall configuration of a virtual counselingsystem according to an embodiment of the present disclosure.

FIG. 2 illustrates a determination unit of a virtual counseling systemaccording to an embodiment of the present disclosure.

FIG. 3 illustrates a text data generation unit of a virtual counselingsystem according to an embodiment of the present disclosure.

FIG. 4 is a flowchart of a virtual counseling method using a virtualcounseling system according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

Advantages, features, and methods of accomplishing the same of thepresent disclosure will become apparent with reference to embodimentsdescribed in detail below together with the accompanying drawings.However, the present disclosure is not limited by embodiments disclosedhereinafter, and may be implemented in various forms. Rather, theseembodiments are provided to so that this disclosure will be through andcomplete and will fully convey the scope of the present disclosure tothose skilled in the technical field to which the present disclosurepertains, and the present disclosure will only be defined by theappended claims. Hereinafter, specific embodiments for carrying out thepresent disclosure will be described in detail with reference to theaccompanying drawings. Regardless of the drawings, like referencenumerals designate like elements, and the term “and/or” includes eachand all combinations of one or more of the associated listed items.

Terms used in the specification are used to describe embodiments of thepresent disclosure and are not intended to limit the scope of thepresent disclosure. In the specification, the terms of a singular formmay include plural forms unless otherwise specified. The expressions“comprise” and/or “comprising” used herein do not preclude the presenceor addition of one or more other elements other than stated elements.

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by thoseskilled in the technical field to which the present disclosure pertains.It will be further understood that terms, such as those defined incommonly used dictionaries, should not be interpreted in an idealized oroverly formal sense unless expressly herein.

Hereinafter, a virtual counseling system and a counseling method usingthe same according to an embodiment of the present disclosure will bedescribed.

FIG. 1 illustrates the overall configuration of a virtual counselingsystem according to an embodiment of the present disclosure. FIG. 2illustrates a determination unit of the virtual counseling system. FIG.3 illustrates a text data generation unit of the virtual counselingsystem.

Referring to FIGS. 1 to 3, the virtual counseling system 100 accordingto an embodiment of the present disclosure may include an input unit110, a determination unit 120, a text data generation unit 130, anoutput unit 140, and a storage unit 150.

The input unit 110 obtains audio information of a user and generatesaudio data. Various input devices may be included to receive data fromthe user.

In other words, the input unit 110 recognizes audio of a user andgenerates audio data. In this case, a device of removing frequenciesother than the human audio frequency or an internal processing step maybe additionally provided, thereby improving an audio recognition rate.

In addition, the input unit 110 may amplify the volume of the audioinformation in order to improve reliability of the user's audio data.Through this, the recognition rate of the user's audio information canbe improved.

Next, the determination unit 120 is connected to the input unit 110 andreceives audio data generated by the input unit 110 to discriminatecharacteristics of the audio data.

In other words, the determination unit 120 receives the audio datathrough the input unit 110, determines a type of the audio data, andgenerates type information on the audio data.

To this end, the determination unit 120 may include a reference unit 121including reference information for determining the type of the audiodata, and an audio classification unit 122 determining the type of theaudio data and generating the type information.

Here, the type information may include first type information includinguser's emotion information and second type information includingsentence-type information of audio data. The sentence-type informationmay be information on whether the audio data generated from the user'saudio information is an interrogative sentence, a plain sentence, aquestion, a request, or a claim.

The reference unit 121 may include learning data serving as referenceinformation for determining to which type the audio data belong. Here,the learning data may include emotional word data for determining auser's emotion and sentence-type data for determining a sentence type ofthe audio data. In other words, the reference unit 121 may includeemotional word data, which is reference data for determining a user'semotion from the user's audio data.

In addition, the reference unit 121 may include sentence-type data,which is reference data for determining a sentence type through thecontent, sound, and tone of the sentence intended by the user from theuser's audio data.

In addition, the reference unit 121 may also include audio-type data fordirectly determining a human audio in addition to the data listed above.Through this, the virtual counseling system according to an embodimentof the present disclosure may ultimately enable machine learning throughhuman produced audio.

As the counseling is repeated and the audio data of users isaccumulated, the contents of the learning data of the reference unit 121are updated. In other words, the contents of the learning data areenriched. As a whole, information of the emotional word data and thesentence-type data is updated through counseling, so that the degree ofcompletion of counseling in the virtual counseling system according toan embodiment of the present disclosure may be improved. This learningmay be carried out by a machine learning method, which is an algorithmof artificial intelligence (AI).

More specifically, as described above, when the learning data of thereference unit 121 is updated, the audio classification unit 122 to bedescribed later may more accurately classify the user's audio throughlearning based on machine learning based on the updated data. In otherwords, based on the updated data of the reference unit 121, the audioclassification unit 122 may expand the audio classification functionthrough learning.

The reference unit 121 may include a storage device to store learningdata.

The audio classification unit 122 compares the audio data and thelearning data of the reference unit 121 with each other, and generatestype information thereby. Alternatively, the audio classification unit122 may generate type information through a system that has been learnedbased on the audio data.

In other words, the audio classification unit 122 compares the user'stone and sound included in the audio data, and the emotional word dataof the learning data with each other to determine the user's emotionalstate.

In addition, the audio classification unit 122 compares the user's toneand speech content (request for explanation, question, request, claim,etc.) included in the audio data and the sentence-type data of thelearning data with each other to determine the sentence type of theuser.

Alternatively, as described above, through the data included in thereference unit 121 or updated data, the audio classification unit 122enables learning for directly classifying the audio data, and thereby,the audio classification unit 122 may perform direct learning, notdependent on data stored in the reference unit 121, and may classify theuser's audio data by type.

In other words, the data of the reference unit 121 may be used asmaterials that the audio classification unit 122 may perform learning bymachine learning, and may not be used as materials that are directlycompared with the user's audio data.

Through this process, the audio classification unit 122 finallygenerates type information. As described above, the type information mayinclude first type information including user's emotion information andsecond type information including sentence-type information of the audiodata. Alternatively, the type information may include only any one ofthe first type information and the second type information.

The audio classification unit 122 of the determination unit 120 mayinclude a control device or a signal processing device to compare theaudio data and the learning data, and thereby generate type information.Alternatively, the audio classification unit 122 may generate typeinformation from audio data based on information and logic acquiredthrough learning. In other words, based on the information acquired bymachine learning, the audio classification unit 122 may generate typeinformation from audio data.

Next, the text data generation unit 130 generates object data byreceiving the type information from the determination unit 120, convertscontent of the audio data into first text data, and combines the objectdata and the first text data to generate second text data.

The second text data generated by the text data generation unit 130 isthe best method. However, in some cases, the text data generation unit130 may not convert the audio data into the first text data, but maydirectly combine the object data and the audio data to form fusion dataand generate the second text data from the fusion data.

To this end, the text data generation unit 130 may include a first textgeneration unit 131 generating the first text data, an object datageneration unit 132 generating the object data, and a second textgeneration unit 133 generating the second text data.

The first text generation unit 131 textualizes the user's speech contentincluded in audio data. To this end, the first text generation unit 131may include audio text information in which audio and text are matchedwith each other. In other words, the first text generation unit 131compares and analyzes the user's audio data and audio text informationwith each other to generate first text data.

When the first text generation unit 131 is implemented based on machinelearning, the first text generation unit 131 generates the first textdata based on the audio data by the learned logic without comparing andanalyzing the audio data and the audio text information.

The object data generation unit 132 generates object data based on thetype information transmitted from the determination unit 120. The objectdata includes the user's emotion information and information on whetherthe content of the user's audio data is an interrogative sentence or aplain sentence.

More specifically, the object data generation unit 132 generates user'semotional information (anger, surprise, question, happiness, thanks,urgency, etc.) based on the first type information among the typeinformation. In addition, the object data generation unit 132 generatesinformation on whether the sentence type is an interrogative sentence ora plain sentence based on the second type information among the typeinformation. The information generated in this way is included in theobject data.

The second text generation unit 133 combines the first text data and theobject data to finally generate second text data. The first text datainclude only the content of speech, but does not expressly include theuser's emotions or intentions. Accordingly, the user's emotion andsentence form (intention), which are object data, are combined with thefirst text data to generate second text data, so that the user'saccurate counseling intention can be grasped.

Emotion information may be displayed in the form of an emoticon in thefirst text data, or a question mark may be included in the first textdata. In this way, the object data is displayed in the first text datain various forms.

The output unit 140 generates feedback data including an answercorresponding to the user's counseling intention based on the secondtext data. The output unit 140 provides the generated feedback data tothe user. The user may check the feedback data to determine whether thecontent of his/her counseling has been correctly performed. If the userdoes not receive satisfactory counseling, the user may receiveadditional counseling by additionally inputting audio informationthrough the input unit 110.

When the user is in an extreme emotional state, virtual counseling maynot proceed smoothly. When the determination unit 120 analyzes theuser's audio data and determines that it is difficult for the user toproceed with a normal counseling (for example, when audio volumeincluded in the audio data is more than a preset decibel (dB)), thevirtual counseling may be stopped, and counseling with an actualcounselor may proceed.

The above-described second text data may be transmitted to a counselor,and the counselor may grasp the counseling intention and emotional stateof the user before the counseling is started based thereon. As a result,the counselor may proceed with preliminary work for smoother counseling,and the user may receive satisfactory counseling.

The information output from the output unit 140 may be stored in thestorage unit 150. When the information is stored in the storage unit150, it is stored by a block chain, so that counseling contents may bestored in a time series. The storage unit 150 may store information suchas contents of a user's counseling, a user's tone during counseling, ahabit, a counseling date and time, and weather. In addition, the storageunit 150 may store the user's counseling content, that is, the user'saudio.

When a user later uses the virtual counseling system 100 of the presentdisclosure, a smoother counseling may be performed by referring to theuser's previous counseling contents based on the stored information.

Next, a virtual counseling method using the virtual counseling system100 according to an embodiment of the present disclosure will bedescribed in detail. FIG. 4 is a flowchart of a virtual counselingmethod using a virtual counseling system according to an embodiment ofthe present disclosure.

As illustrated in FIG. 4, the virtual counseling method S100 using thevirtual counseling system according to an embodiment of the presentdisclosure includes an input operation S110, a determination operationS120, a text data generation operation S130, and an output operationS140.

In the input operation S110, audio data is obtained from a user. Inother words, the user inputs audio information and converts the audioinformation into an electric signal to obtain audio data. In thisoperation, in order to improve the degree of extraction of audio data,it may be possible to amplify the user's audio information.

Subsequently, in the determination operation S120, the audio data isreceived, the type of the audio data is determined, and type informationon the audio data is generated.

The type information is generated by the above-described determinationunit (120 in FIG. 1), wherein the determination unit 120 may include areference unit (121 in FIG. 2) including reference information fordetermining the type of the audio data, and an audio classification unit(122 in FIG. 2) determining the type of the audio data and generatingthe type information.

Here, the type information may include first type information includinguser's emotion information and second type information includingsentence-type information of the audio data.

The reference unit 121 may include learning data serving as referenceinformation. Here, the learning data may include emotional word data fordetermining a user's emotion and sentence-type data for determining asentence type of the audio data. The information of the emotional worddata and the sentence-type data may be updated through repeatedcounseling. In addition, the learning data may include the user's audio.Accordingly, the audio classification unit 122 may perform learningthrough the learning data of the reference unit 121. In other words, theaudio classification unit 122 may determine the contents of the user'scounseling from the user's audio data from the learning data by machinelearning.

In the text data generation operation S130, the object data is generatedby receiving the above-described type information, the content of theaudio data is converted into first text data, and the object data andthe first text data are combined to generate second text data. Here, theobject data may include the user's emotion information and informationon whether a content of the user's audio data is an interrogativesentence or a plain sentence.

In the output operation S140, feedback data is generated based on theabove-described second text data, and the feedback data is provided tothe user, so that the user may receive a reply to the inquiredcounseling content.

For reference, if the feedback data generated based on the second textdata is provided to the user, but the user does not receive satisfactorycounseling, the user may additionally input audio information throughthe input unit and receive additional counseling.

Through the input of additional audio information, the amount ofinformation of the above-described learning data increases, and theamount of learning of the audio classification unit 122 is increased bythe increased amount of information. Accordingly, the audioclassification unit 122 would have the ability to more accuratelydetermine the type of the user's audio data.

When it is determined that it is difficult to proceed with a virtualcounseling with a user, the second text data is transmitted to acounselor, and the counselor may consult with the user based on thesecond text data.

Specifically, individual weights may be applied to each sentence, word,context, emotion, etc. based on the first text data or object dataincluded in the second text data generated during the virtual counselingwith the user. It may be possible to calculate numerically how close tothe threshold for situations in which a response is needed.

In addition to virtual counseling in advance, a complaint index may beprovided to evaluate whether a professional counselor should conductcounseling over a direct phone call. The complaint index establishes aseparate database for each sentence and word in advance. When acorresponding sentence or word appears directly or a similar wordappears, a complaint index, which is the sum of the sub-complaintindexes, may be determined by assigning a predetermined weight to thecorresponding sub-complaint index.

A high weight may be applied to words or sentences that directlyindicate customer complaints. If the value of the complaint index, whichis the sum of each sub-complaint index, exceeds a predeterminedthreshold while virtual counseling continues, the complaint counselorwho is currently available and has the highest matching rate among theresponders in the complaint response team database stored or managed inadvance is matched. The corresponding complaint counselor may accuratelyrecognize the previous virtual counseling situation based on thetransmitted second text data, and may continuously respond. In thisprocess, the matching rate of the complaint counselor may be calculatedaccording to the type of the second text data classified according to apredetermined criterion in addition to the sum of the complaint indexes.To this end, the second text data may be assigned a type according to apredetermined classification. The matching rate of the complaintcounselor may be determined in further consideration of whether thecounselor has an experience in responding to similar complaintsituations in the past, and may be determined based on the similaritywith the type of the second text data described above. To this end, itmay be possible to collect and manage counseling history data for eachcomplaint counselor, and by analyzing the personality and behavior typeof each complaint counselor in advance, it may be possible to determinethe complaint counselor who better matches the situation, the intensityof the complaint, the topic of the complaint, and the reason for thecomplaint.

From the foregoing, although the embodiments of the present disclosurehave been described. It should be noted, however, that the presentdisclosure is not limited to the above embodiments, and may beimplemented in various forms. Those skilled in the technical field towhich the present disclosure pertains will understand that the presentdisclosure may be practiced in other detailed forms without departingfrom the technical spirit or essential features of the presentdisclosure. Therefore, it should be understood that the above-describedembodiments are exemplary in all aspects rather than being restrictive.

1. A virtual counseling system, comprising: an input unit obtainingaudio information from a user and generating audio data; a determinationunit receiving the audio data through the input unit, determining a typeof the audio data, and generating type information on the audio data;and a text data generation unit generating object data by receiving thetype information from the determination unit, converting content of theaudio data into first text data, and combining the object data and thefirst text data to generate second text data.
 2. The virtual counselingsystem of claim 1, wherein the determination unit comprises a referenceunit including reference information for determining the type of theaudio data, and an audio classification unit determining the type of theaudio data and generating the type information.
 3. The virtualcounseling system of claim 2, wherein the type information comprisesfirst type information including emotion information of the user andsecond type information including sentence-type information of the audiodata.
 4. The virtual counseling system of claim 2, wherein the referenceunit comprises learning data serving as the reference information. 5.The virtual counseling system of claim 4, wherein the learning datacomprises emotional word data for determining the user's emotion andsentence-type data for determining a sentence type of the audio data. 6.The virtual counseling system of claim 5, wherein information of theemotional word data and the sentence-type data is updated throughrepeated counseling.
 7. The virtual counseling system of claim 1,wherein the text data generation unit comprises a first text generationunit generating the first text data, an object data generation unitgenerating the object data, and a second text generation unit generatingthe second text data.
 8. The virtual counseling system of claim 7,wherein the object data comprises the user's emotion information andinformation on whether a content of the user's audio data is aninterrogative sentence or a plain sentence.
 9. The virtual counselingsystem of claim 1, wherein feedback data generated based on the secondtext data is provided to the user, and when the user does not receivesatisfactory counseling, the user additionally input audio informationthrough the input unit.
 10. The virtual counseling system of claim 1,wherein the input unit amplifies a volume of the audio information inorder to improve reliability of the user's audio data.
 11. A virtualcounseling method performed by a system, the method comprising:obtaining audio data from a user; receiving the audio data, determininga type of the audio data, and generating type information on the audiodata; generating object data by receiving the type information,converting content of the audio data into first text data, and combiningthe object data and the first text data to generate second text data;and generating feedback data based on the second text data and providingthe feedback data to the user.
 12. The method of claim 11, wherein thedetermination unit of the system generates the type information, andwherein the determination unit comprises a reference unit includingreference information for determining the type of the audio data, and anaudio classification unit determining the type of the audio data andgenerating the type information.
 13. The method of claim 12, wherein thetype information comprises first type information including emotioninformation of the user and second type information includingsentence-type information of the audio data.
 14. The method of claim 12,wherein the reference unit comprises learning data serving as thereference information.
 15. The method of claim 14, wherein the learningdata comprises emotional word data for determining the user's emotionand sentence-type data for determining a sentence type of the audiodata.
 16. The method of claim 15, wherein information of the emotionalword data and the sentence-type data is updated through repeatedcounseling.
 17. The method of claim 11, wherein the object datacomprises the user's emotion information and information on whether acontent of the user's audio data is an interrogative sentence or a plainsentence.
 18. The method of claim 11, further comprising: transmittingthe second text data to a counselor when it is determined that it isdifficult to proceed with a virtual counseling with the user, whereinthe counselor consults with the user based on the second text data.